A CLASS OF MARKOV CHAINS WITH NO SPECTRAL GAP 

YEVGENIY KOVCHEGOV AND NICHOLAS MICHALOWSKI 



Abstract. In this paper we extend the results of the research started in [6] 
and [7], in which Karlin-McGregor diagonalization of certain reversible Markov 
chains over countably infinite general state spaces by orthogonal polynomials 
was used to estimate the rate of convergence to a stationary distribution. 

We use a method of Koornwinder [5] to generate a large and interesting 
family of random walks which exhibits a lack of spectral gap, and a polynomial 
rate of convergence to the stationary distribution. For the Chebyshev type 
subfamily of Markov chains, we use asymptotic techniques to obtain an upper 
bound of order O { ^°^ ^ ^.nd a lower bound of order O on the distance 

to the stationary distribution regardless of the initial state. Due to the lack 
of a spectral gap, these results lie outside the scope of geometric ergodicity 
theory [8]. 



1. Introduction 

Let P = [p{i,j) ) be a reversible Markov chain over a sample space fi, that is, 
it must satisfy the following detailed balance conditions: 

where tt is a non-trivial non- negative function over £7. If P admits a unique sta- 
tionary distribution then tt = v. 

It can be shown that the reversible P is a self-adjoint operator in ^^(tt), the space 
generated by the following inner product induced by tt 

If P is a tridiagonal operator (i.e. a nearest-neighbor random walk) on £7 = {0, 1,2,.. 
then it must have a simple spectrum, and is diagonalizable via orthogonal polyno- 
mials as it was studied in the 50's by Karlin and McGregor, see [3], [TO], and [5]. 
There, the extended eigenfunctions Qj{X) satisfying Qo = 1 and 
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are orthogonal polynomials with respect to a probability measure ip. If we let pt (i, j) 
denote the entries of the operator P* that represent t step transition probabilities 
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from state i to state j then 



1 

t/ 



Pt{i,j) = ^3 j ^ A'Q.(A)g,(A)d^(A) V*, J G 
where ttj with ttq = 1 is the reversibility measure of P. 

We win use the following distance to measure the deviation from the stationary 
distribution on a scale from zero to one. 

Definition 1. If /x and v arc two probability distributions over a sample space f2, 
then the total variation distance is 

Let p = X^fco""*:- Observe that p < oo if and only if the random walk P is 
positive recurrent. Recall that v — ^tt \s the stationary probability distribution. 
If in addition to being positive recurrent, the aperiodic nearest neighbor Markov 
chain originates at site j , then the total variation distance between the distribution 
j-it = fJ'oP* a-iid v is given by 

(1.1) ||:.-^,||^^ = i^7r„ / A*g,(A)Q„(A)#(A) 

as measure ip contains a point mass of weight i at 1. See [6]. 

The rates of convergence are quantified via mixing times, which for an infinite state 
space with a unique stationary distribution arc defined as follows. Here the notion 
of a mixing time depends on the state of origination j of the Markov chain. See [7] . 

Definition 2. Suppose P is a Markov chain with a stationary probability distri- 
bution ly that commences at Xq ~ j. Given an e > 0, the mixing time t^ixi^) is 
defined as 

tmixi^) = uim{t : Wv - fJ-tWrv < e} 



In the case of a nearest-neighbor process on fi = {0, 1,2,...} commencing at j, 
the corresponding mixing time has the following simple expression in orthogonal 
polynomials 



tmixie-) = min < ^ : ^ 7r„ 



X'QjiX)Qn{X)d^P{X) 
(-1.1) 



< 2e 



Investigations into the use of orthogonal polynomial techniques (see [3], |10| ) in 
the estimation of mixing times and distance to the stationary distribution has been 
carried out in [7] for certain classes of random walks. In this paper we consider 
the problem from the other direction. Namely given a large class of orthogonal 
polynomials we outline how to find the corresponding random walk and estimate 
the rate for the distance to the stationary distribution. 

More specifically beginning with the Jacobi polynomials, whose weight function lies 
in (—1, 1) we use Koornwinder's techniques [5] to attach a point mass at 1. For 
the class of Jacobi type polynomials Qn thus obtained, the three term recurrence 
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relationship is understood j4]. The tridiagonal operator corresponding to these 
polynomials is not a Markov chain, however the operator can be deformed to become 
one. The corresponding changes in the polynomials arc easy to trace. This gives 
a four parameter family of nearest neighbor Markov chains whose distance to the 
stationary distribution decays in a non-geometric way. In principle the asymptotic 
analysis presented in this paper can be applied to the entire four parameter family. 
Wc outline how this proceeds for Chebyshev-type subfamily consisting of taking 
a = /? = — 1/2 in the Koornwinder class. 

Wc would like to point out the important results of V. B. Uvarov |llj on transfor- 
mation of orthogonal polynomial systems by attaching point masses to the orthog- 
onality measure, predating the Koornwinder results by fifteen years. The results of 
V. B. Uvarov can potentially be used in order to significantly extend the scope of 
convergence rate problems covered in this current manuscript. 

The paper is organized as follows. In Section [2] we discuss constructing positive 
recurrent Markov chains from the Jacobi family of orthogonal polynomials adjusted 
by using Koornwinder's techniques to place a point mass at x — 1. Next, we 
derive an asymptotic upper bound on the total variation distance to the stationary 
distribution in the case of general a > — 1 and /? > — 1 in Section [31 Our main 
result. Theorem [21 is presented in Section [31 There, for the case of Chebyshev type 
polynomials corresponding to a = /3 = ^1/2, we produce both asymptotic lower 
and upper bounds for the total variation distance. Finally, in Section[5lwe compare 
our main result to related results obtained by other techniques. 



2. From Orthogonal Polynomials to Random Walks via Koornwinder 



T. Koornwinder [5] provides a method for finding the orthogonal polynomials 
whose weight distribution is obtained from the standard Jacobi weight functions 
Ca.pi^ — x)"{l + x)^ by attaching weighted point masses at —1 and 1. A spectral 
measure corresponding to a Markov chain contains a point mass at —1 if and only 
if the Markov chain is periodic. A spectral measure for an aperiodic Markov chain 
contains a point mass at 1 if and only if it is positive recurrent. Thus in order to 
create a class of positive recurrent aperiodic Markov chains with a Koornwinder 
type orthogonal polynomial diagonalization we will only need to attach a point 
mass at 1 and no point mass at —1. 



Let iV > and let a, /3 > -1. 

{a + 



(2.1) 
where 



For n = 0,1,2, 
/3 + 2)„_i 



, . define 

- A^(l + 
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(a + l)n 
(/? + l)n ' 



n{n + a + (3 + 1)N 



{a + l)nia + P + 2).a-i (Q' + 1) 

P"'^ is the standard Jacobi polynomials of degree n and order (a,/3), {x)n = 
a;(x-|-l)---(2;-|-n— 1). These polynomials form a system of orthogonal polynomials 

with respect to the probability measure dil}{x) = ^jy+i where 

Ca,p = B{a+i p+i) ' '^(■'■) the beta function, and 5i{x) denotes the a unit 
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point mass measure at x = 1. See T. Koornwiiider [5]. Direct calculation shows 
that P^''''^(l) = ^2±l)iL^and so we normalize Q„(a;) = n\P^^^^^{x)/{a + l)n which 
is the orthogonal set of polynomials with respect to d4> satisfying Qn(l) = 1. 

As we have mentioned earlier, the tridiagonal operator H corresponding to the 
recurrence relation of the orthogonal polynomials may not be a Markov chain op- 
erator. Let Pi, Vi and qi denote the coefficients in the tridiagonal recursion 

PiQi+i{x) + nQi{x) + q.iQi^i{x) ^ xQ^{x), 

for i = 0, 1, 2, . . . , where we let Q^i = as always. 

Notice because the polynomials are normalized so that Qi{l) = 1 it follows imme- 
diately that Pi + ri + Qi = 1. However some of the coefficients pi, ri, or qi may turn 
out to be negative, in which case the rows of the tridiagonal operator A would add 
up to one, but will not necessarily consist of all nonnegative entries. 

In the case when all the negative entries are located on the main diagonal, this 
may be overcome by considering the operator {H + A/) . For A > — inf this 

ensures all entries in the matrix -^^{H + A/) are nonnegative and hence can be 
thought of as transition probabilities. More generally, if a polynomial p{-) with 
coefficients adding up to one is found to satisfy p{H) > coordinatewise, then such 
p{H) would be a Markov chain. 



3. An Asymptotic Upper Bound for Jacobi type Polynomials 

In this section we derive asymptotic estimates for the distance to the stationary 
distribution when our operator given by P\ = j^{H + XI) is a Markov chain. In 

this case the Karlin-McGregor orthogonal polynomials for Px are Qj ^(1 + A)a; — A 

and the orthogonality probability measure is j^d'ip(^{l + X)x — A^ over (^j^, 1 
where the Qj are the Jacobi type polynomials introduced by Koornwinder from the 
previous section. 

Of course the new operator Px is again tridiagonal. For the n-th row of Px, let us 
denote the {n — l)-st, n-th, and (n + l)-st entries by q^, r^, and p^ respectively. 
Here the entries of Pa can be expressed via the entries of H as follows 



Pn = -T— T' = -T— T' ^nd g„ = 



1 + A' " 1 + A' ^" 1-t-A 

Clearly we still have that + t"^ + 9,^ = 1- 

With the probabilities in hand we now compute the corresponding reversibility 
function tt^ of Pa which is equal to the corresponding function of H defined as 



^ Po-P^-i _ jjere tto^ = 1 = and tt,-^ = = ^"-"-^ = 

Changing variables in (jl.ip yields 



\\v- litWrv ^X]'^'' 

n=0 



(-1,1) 



1+A 1+A 



A \* 

Qj{x)Qn{x) d^p{x) 
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Lemma 1. Consider the case when pn > and Qn > for all n ^ 0, dJid 
oo > A > — inf Ti . Then, for the Jacobi type polynomials Qj the distance to the 

i 

stationary distribution satisfies the following bound 

(3.1) ii--MdU<%^;^|^E-niiQ«iioo + J E 

^ ' n=0 n=j+t+l 

for a certain constant Ca,p,x- 



Proof. For n > j + t, it follows from the orthogonality of the polynomials and our 
normalization Qi{l) = 1 that 



A \* 



?j(a;)Qn(a;) dip{x) = 1 



It is then easy to sec that — < I + II + J2'^=j+t+i where 

'x + X\t 



^ n=0 -^(-1^0) 
j+t 



1 + A 



Qjix)Qn{x) 



and II=]:i2^nl (^^)*|Q,(x)Q„(a;)|(l ~ + x)^ 

^„=o -'(0,1) + 



To estimate / notice that 



x+\ 
1+A 



< max( ^ 



1+A' 



l-A 



1 + A 



< 1 for A > 0. Hence 



I < Aj{\t\)e ^* for an appropriate polynomial Aj{-) such that 



. j+t 



{l-xYil^xf dx<A.j{\t\), 

'(-1,0) 

and c = — log|max(Y^, )|- Such polynomial Aj exists since ||Q„||oo grows 
polynomially in n and 7r„ is bounded. See formula 22.14 A in Abramowitz and 
Stegun [T]. 

Thus / is clearly bounded by the right hand side of (|3.ip . 

For the second term, II < ^ Ei=o WQnQjW^ /p (f$i)*(l - ^Til + ^Y' dx. 
There we make the change of variables s = — log(Y^), and for simplicity let 
x{s) = (1 + A)e^''* — A. Then the integral reduces to 

.log(i±A) 



.-s(t+i) 



(l-A + (l + A)e-*)'^(l-e-") 



Using the fact that (l-e"'')" = s"(^l + 0(s)) and (l-A+(l + A)e-'')'3 = 2'3 + 0(s), 
the above integral becomes 



(1 + A) 



l+a 



log(J 



-s{t+ 



i'(2^s" + 0(s"+i)) ds, 
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where the upper bounds 0{s) can be made specific. Next, applying the standard 
asymptotic methods of Laplace to this yields the following asymptotics 

A (( + !)■+'> 

Thus one can obtain a large enough constant Ca,p,x such that 



□ 



In order to derived effective bounds on \\v — iitWrv '^^ '^^ necessary to gain a more 
detailed understanding of 7r„ and HQnlloo- When min(a,/?) > — the ||Qn|loo can 
be estimated using the known maximum for the Jacobi polynomials found in Lemma 
4.2.1 on page 85 of [2] together with Koornwinder's definition of these polynomials. 

One way to derive estimates for TTn is to use the expression 7r„ in terms of p„, r„, 
and qn- For Koorwinder's class of polynomials these expressions are derived for all 
a,f3,M,N in [3]. It can be verified directly that in the case when M = 0, then 

Po 



{i+N){a+0+2} ^fter taking into account the normilization (5„(1) = 1, 

and taking into account a small typo, it can be verified from equations (41)-(45) 
in [4] that p„ and q„ are positive for n > 1. Thus the conditions for Lemma [Hare 
satisfied for all a, f3 > —1. Furthermore, from (18), (19) and (32) in U it can be 
easily seen that p„ — )■ i and — > ^ as n — oo, and hence r„ = 1 — p„ — q„ — >■ as 
n — >■ oo. Thus for A large enough the operator corresponds to a Markov chain. 

As the expressions for these quantities laborious to write down, instead we focus our 
attention on a specific case in which our calculations are easy to follow. Specifically 
we focus on the Chcbyshcv polynomials. 



4. Chebyshev Polynomials: Upper and Lower Bounds 

By applying Koorwinder's results to the Chebyshev polynomials of the first kind 
which correspond to the case oi a = (3 = we arrive at a family of orthogonal 

polynomials with respect to the measure ^—^l^^dx + Ndi{x)j . Using (|2.ip 
we find that here, 

g„(x) := -Nix + l)Un-iix) + (1 + 2nN)T^{x), 

where T„ and C/„ denote the Chebyshev polynomials of the first and second kind 
respectively. Notice that C/ri(l) = n + 1 and r„(l) = 1, which immediately to verify 
that Q„(l) = 1. 
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Once again we consider the operator 
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on £'^{tt), so that vector {Qo{x),Qi{x),Q2{x), . . .)'^ is an eigenvector with eigen- 
value X. 

Specifically the numbers p„, r„, and g„ satisfy poPi{x) + rQPQ{x) = a; for n = 0, 
and 

(4.1) PnQn+lix) + TnQnix) + qnQ„-l{x) ^ xQnix) forn>l. 

Keisel and Wimp [4| give expressions for p„, Vn and g„ for n > 0. To find the expres- 
sions directly in this case one could use (|4.1|) to derive three linearly independent 
equations, and solve for p„, r„, and g„. 

For the case n ~ Q the equation immediately gives us that po — ivTT ~ 
jf^- Evaluating at convenient choices of x, such as —1, 0, 1, do not yield linearly 
independent equations for all n. One solution to this is to evaluate at x = 1,-1 and 
differentiate (|4.1[) and then evaluate at a; = 0. This gives three linearly independent 
equations and a direct calculation then shows that 



(4.2) 



Pn = 7^ 



1 1 + (2?i - 1)7V 
1 + (2n+ l)iV' 



9« = ^ 



1 1 + (2?i + l)N 
l + (2n-l)iV' 



and 



{l + {2n-l)N){l + {2n+l)N) 



As r„ < the operator H fails to correspond to a Markov chain. However this is 
the case we addressed at the end of Section [2] of the current paper. Thus consider 
Pa = TTxi-^ ~^^^)- Now, since |r„| is a decreasing sequence for n > 1. So provided 
that A > |ri| = (i+N)(i+d.N) ' then have p^,r^,q^ > 0. Thus we can consider 
these coefficients , r,^ , and as the transition probabilities in a nearest neighbor 
random walk. 

Recall that tt,^ = 7r„ = . Thus for Px we can directly calculate 7r„ from (|4.2p . 

We have that po---Pn-i = 2^ i+(27^-i)JV similarly gi • • • q„ = ^ ^^''1" jv^''^ ■ 



Thus TTn — 



2(l+jV)jV 



(l + (2n-l)A')(l + (2n+l)Ar) ' 
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Theorem 2. Given N > and A > [l-^-N)^l-^3N) • Consider the case of the 
Chebyshev-type random walks over Q = {0, 1,2,...} with probability operator 
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where - ^ ■ „A _ i . i+(2n+i)iv ^ r^ - I - - 

"""^"^ ~ 2(1+A) l + (2n+l)Af' ~ 2(1+A) l + (2ri-l)W ' n ^ l^n Hn 

for n > 1, with p^ = (i_^;,)(jv+i) = 1 - '"o • 

Then for the random walk originating at some site j G fl, there are positive con- 
stants c and C that depend on j, N and A such that 

for t sufficiently large. 



Proof. For the upper bound we simply need to estimate the sums appearing in 
Lemma[T] Since 7r„ = 0( i^„^iyi ) , it is easy to see that the second sum X]J^j+t+i """n 
is bounded by C'n /{t + j + 1). The main term turns out to be the first sum. 

In the case of the Chebyshev type polynomials we have the bound HQnllo^ — 4A^n + 1. 
Thus the first sum in Lemma[T]is bounded by Caj3.x,N ^ for an appropriate 

constant Ca^p^x^N. And so, for an appropriate C and large i, 

II II 



On the other hand, recalling that Qoix) = ttq = 1, we have that: 

'a; + A\* 



, > 



1,1) ^ + 



(-1,1) 



However we have already shown that for large enough t, the above right-hand side 
is asymptotic to -S 



+t 



□ 



We finish with some concluding remarks. At first the bound HQnlltxi — 4A^n + 1 
may appear somewhat imprecise since near a; = 1, we have that Q„(l) = 1. It 
is tempting to suggest that the correct asymptotic for the total variation norm is 
C l\ft. However on closer examination in the neighborhood of a; = 1, « v? . 

This V? causes the errors to be at least of the order of the main term. Over- 
all it seems unlikely to the authors that Cjyft is the correct asymptotic for the 
Chebyshev-type polynomials. 
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5. Comparison to other methods 



An ergodic Markov chain P ~ (pih j) with stationary distribution v is said 

to be geometrically ergodic if and only if there exists < i? < 1 and a function 
M : 57 — 7> M+ such that for each initial state i € the total variation distance 
decreases exponentially as follows 

ibi(z, •) - v{-)\\tv = \y. \pt{i,]) - < 

jen 

In other words, an ergodic Markov chain is geometric when the rate of convergence 
to stationary distribution is exponential. See [S] and references therein. 

If the state space Q. is finite, \Q\ ^ d < oo, and Markov chain is irreducible and 
aperiodic, then P will have eigenvalues that can be ordered as follows 

Ai = 1 > IA2I > ••• > lArfl 

In which case, the Perron-Frobenious Theorem will imply geometric ergodicity with 

iiPt(*,-)-K-)iiw=o(t™=-i|A2r), 

where TO2 is the algebraic multiplicity of A2. Here the existence of a positive spectral 
gap, 1— IA2I > 0, implies geometric ergodicity with the exponent — log IA2I ~ 1— IA2I 
whenever the spectral gap is small enough. 

When dealing with Markov chains over general countably infinite state space 11, 
the existence of a positive spectral gap of the operator P is essentially equivalent to 
the chain being geometrically ergodic. For instance, the orthogonal polynomial ap- 
proach in [7] resulted in establishing the geometric rate R ~ max |r + l^fpq, q+r } 
for the Markov chain 
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q > p, r > 
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over D, 



together with establishing the value of the spectral gap, 1 — ?- > 0. 

As for the Markov chain Pa considered in Theorem [5] of this paper, its spectral 
measure j^dip(jll + X)x — Xj over ^j^, 1 admits no spectral gap between the 
point mass at 1 and the rest of the spectrum implying sub-geometric ergodicity. The 
sub-exponential rate in total variation norm is then estimated to be of polynomial 
order between -K= and 

In the field of probability and stochastic processes, there is a great interest in finding 
methods for analyzing Markov chains over general state space that have polynomial 
rates of convergence to stationary distribution. In Menshikov and Popov [S] a one 
dimensional version of Lamperti's problem is considered. There, a class of ergodic 
Markov chains on countably infinite state space with sub-exponential convergence 
to the stationary probabilities is studied via probabilistic techniques. One of their 
results relates to our main result. Theorem [2] Namely, Theorem 3.1 of [9] when 
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applied to our case, implies for any £ > the existence of positive real constants 
Ci and C2 such that 

Thus for the Markov chain considered in Theorem [5J the orthogonal polynomials 
approach provides a closed form expression for the difference v — ^t, and a signif- 
icantly sharper estimate on convergence of fit to the stationary distribution i/, for 
both the single state distance \iy{0) — /it(0)| and a much stronger total variation 
norm, - HtWrv- 
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